In some conventional data storage systems, such as those employing hard disk drives, compact disks, digital versatile disks (DVDs), flash memories, and random access memories, the so-called user data and metadata can be stored in a common sector. The user data can include data such as, for example, documents, photographs, audio files, media files, operating files, and the like. Additional information about the user data can be generated by the system and may correspond to, for example, the length and/or location of the user data, the time that the user data was created, the type of user data, the protection status of the user data, and the like. This additional information is sometimes referred to as system data, or metadata. In general, the user data occupies significantly more storage space than does the corresponding metadata. To facilitate system operations such as data organization, indexing and searching, storage systems often need access only to the metadata.
Some conventional data storage systems may also incorporate error correction and/or error detection to ensure reliable storage and retrieval of both the user data and metadata. For example, redundancy data may be generated from the corresponding user data and/or metadata by employing error correction algorithms such as Bose-Chaudhuri-Hocquenghem (BCH) encoding, Reed-Solomon encoding, Low-Density Parity-Check encoding, Hamming encoding, Reed-Muller encoding, binary Golay encoding, Viterbi encoding, Turbo encoding, Optimal Rectangular encoding or iterative encoding, or by employing error detecting algorithms such as parity bits, checksums, or Cyclic Redundancy Checking. Redundancy data is typically stored in the same sector as the user data and metadata.
Referring to the examples of FIGS. 1 and 2, some conventional architectures include storing user data 20, metadata 30, and redundancy data 40 in the same sector 10 of a storage system, where redundancy data 40 corresponds to jointly encoded user data 20 and metadata 30. Jointly encoding the user data and metadata (i.e., encoding the user data and metadata together as a single serial data stream) provides for more reliable storage and retrieval of the user data. Assume for purposes of illustration and without limitation, that in each sector 10, 4 kilobytes are allocated to user data 20 and 80 bytes are allocated to metadata. If redundancy data 40 is generated corresponding to 16-bit BCH encoding of user data 20 and metadata 30 jointly, it is observed that, when the BCH error correction power (T) equals 60 (corresponding to about T*16 bits of binary data) are allocated to redundancy data 40, the failure rate probability of decoding both the user data and the metadata is about 10^−5. Similarly, when T=117 (corresponding to about 117*16 bits of binary data) are allocated to redundancy data 40, the failure rate probability of decoding both the user data and the meta data is about 10^−25. However, because redundancy data 40 is generated from both user data 20 and metadata 30, both user data 20 and metadata 30 must be read and jointly decoded, even when it desired only to retrieve the metadata 30. Thus, some conventional architectures, while enabling superior reliability of data storage and retrieval, do not enable high throughput retrieval of the metadata alone.
Thus, in some other conventional architectures, two sets of redundancy data are provided—one corresponding to the user data alone and one corresponding to the metadata alone. In these schemes, the metadata may be decoded with its corresponding redundancy data independently of the user data, thus enabling higher throughput retrieval of the metadata alone. Referring to the examples of FIGS. 3 and 4, user data 120, metadata 130, first redundancy data 143 corresponding to user data 120, and second redundancy data 146 corresponding to metadata 130 are stored in the same sector 110 of a storage system. However, the total space allocated to redundancy data is now shared between first redundancy data 143 and second redundancy data 146, which may increase the overhead cost and/or reduce the capabilities of error checking and/or correction (ECC).
Consider again the example where the redundancy data is generated corresponding to 16-bit BCH encoding, 4 kilobytes are allocated to user data 120, 80 bytes are allocated to metadata 130, and the total available space allocated for the encoded redundancy data (i.e., first redundancy data 143 and second redundancy data 146) is 220 bytes. In other words, if 16-bit BCH encoding is used, the total error correction power T_total=110. As illustrated, when a greater number of bits are allocated to second redundancy data 146, a lesser number of bits are available for allocation to first redundancy data 143, therefore resulting in a higher failure rate probability of user data 120. If it is desired for the metadata to have a failure rate probability less than or equal to the user data failure rate probability, the respective allocation of bits to the first and second redundancy data must satisfy (e.g., be at or to the left of) solution line 180. Thus, for error checking and/or correction of metadata 130 to be as reliable as user data 120, no more than about T=95 (corresponding to 95*16 bits) must be allocated to first redundancy data 143 (see, e.g., solution point 183). As a result, about T=15 (corresponding to 15*16 bits) are available for allocation to second redundancy data 146 (e.g., the difference between the total ECC correction power [117] and the ECC correction power allocated to the first redundancy data [95]). In this case, the failure rate probability of the user data is about 10^−17 (in contrast to 10^−25 for the joint encoding architecture of FIGS. 1 and 2). Therefore, while independent encoding of user data and metadata provides for higher throughput retrieval of metadata alone, it does not provide superior reliability of data storage and retrieval.
On the other hand, increasing the number of ECC bits to separately error check and/or correct decoded user data and metadata would not only increase the amount of memory consumed, but also the complexity of the error checking and/or correction logic. A need therefore exists to provide for fast, independent, and reliable retrieval of metadata alone wherein there is minimum degradation in the reliability of user data.